Cameras have a long history as means for recording visual information. In recent years, in place of silver-halide cameras for taking photographs by using a film and a photosensitive plate, digital cameras for digitizing an image by using a solid-state image-capturing device, such as a CCD (Charge Coupled Device) or a CMOS (Complementary Mental-Oxide Semiconductor) have become widely popular. Digital cameras have advantages as follows. Digitally coded images can be stored in a memory, image processing and image management using a computer can be performed, and there is no problem regarding the lifetime of a film.
A CCD sensor is configured by a scheme in which pixels (photodiodes) arranged in two dimensions convert light into electric charge by using a photoelectric effect. On the surface of each pixel, for example, a color filter of one of three colors of R (red), green (G), and blue (B) is provided, and signal electric charge corresponding to the amount of incident light through each color filter is stored. Then, signal electric charge corresponding to the amount of incident light of each color is read from each pixel, and the color of the incident light at each pixel position can be reproduced on the basis of each amount of each signal electric charge of the three colors. However, there is an upper limit in the amount of signal electric charge that can be stored by each pixel. There is a limitation that signal electric charge exceeding the amount of saturation electric charge of photodiodes constituting a pixel cannot be stored. That is, the range from the noise level possessed by a sensor device to the amount of saturation of electric charge that can be stored by the sensor device becomes a dynamic range. An image in a portion outside the dynamic range is observed as a phenomenon such that the captured image becomes overexposed or underexposed.
The human eye can sense brightness in correspondence with the number of digits of the amount of light, and the range, that is, the dynamic range, is said to be 8 digits. The dynamic range of a negative film is at least 4 digits. On the other hand, the dynamic range of a CCD is, for example, approximately 2.4 digits, and is insufficient. In recent years, with the advancement of microfabrication technology, the number of pixels in a solid-state image-capturing device has markedly increased. With the resolution power nearing that of silver-halide photographs, the amount of saturation electric charge has decreased with pixel size, and the problem of an insufficient dynamic range has become more serious.
A camera has a function of correcting the amount of light being received through the aperture and the shutter speed. For example, when the aperture is shut by one stop, the amount of light entering the camera is halved. If the amount of light that enters the camera is halved, the amount of light can be moved in parallel by the amount of 0.5 with respect to the logarithmic scale. However, the correction of the amount of light by the camera allows the distribution of the amount of light of a subject only to be moved in parallel within the dynamic range, and has no meaning with respect to the distribution of the amount of light wider than the dynamic range.
There is a known method of increasing the dynamic range by performing multiple recording and combining of images by changing the shutter speed. That is, under the control of an electronic shutter in a CCD sensor, an image in which saturation is avoided in a high-speed shutter process, and an image of a low-luminance portion made up of a sufficient number of electrons in a low-speed shutter process are obtained and combined, thereby ensuring a satisfactory dynamic range. However, image capturing needs to be performed two times, that is, a high-speed shutter process and a low-speed shutter process have to be performed, in order to obtain one image. Therefore, it is not possible to obtain an appropriate still image with respect to a moving object. Also, since a double image-capturing system becomes necessary, the configuration of the apparatus becomes complex.
Another method is considered in which sensors having a high sensitivity and sensors having a low sensitivity are alternately arranged in the sensor section, and a dynamic range is ensured by combining the information of the two types of sensors. However, there is a problem in that the sensor configuration and signal processing of sensor output becomes complex.
For example, a signal processing method for detecting the luminance distribution of a captured image, correcting the luminance distribution in which back light, white color loss, and underexposure occurred to an optimum luminance distribution, and outputting it has been proposed (refer to, for example, Patent Document 1). According to the signal processing method, when the luminance distribution of an output image is large, correction is made using a B-spline interpolation method so that the luminance distribution becomes uniform, making it possible to control the input/output of γ correction and to obtain an optimum image.
Furthermore, an automatic exposure control method (refer to, for example, Patent Document 2) has been proposed in which, when a high-luminance subject signal contained in an input image signal is detected, the amount of exposure used for an image-capturing operation is controlled in response to the high-luminance subject signal and also control of increasing the luminance level in a low-luminance region with respect to the amount of γ correction for the image signal is performed on the basis of the amount of exposure, thereby a satisfactory image is obtained, for example, from a video image in which a high-luminance subject and a low-luminance subject coexist in a back light state.
Furthermore, a method of ensuring a dynamic range of an image signal using γ correction is also considered. γ correction refers to a process in which, when a captured image is to be output to a monitor or the like, the relationship between data of colors of an image or the like and signals when the data is actually output is adjusted, and gradation (luminance) is correctly reproduced (as a more nature display). Gradation (luminance) characteristics on a monitor are not linear with respect to the level of an input signal from the CCD. If the input signal is denoted as x and the luminance level of a monitor is denoted as y, the relation of y=xγ holds. Accordingly, γ correction performs a process corresponding to this reverse function. A γ correction curve is primarily a function of a power, and a dynamic range can be ensured by changing a γ coefficient. However, there is a problem in that, in a captured image in a portion that has been separated from an ideal γ curve as a result of the change in the γ coefficient, negative effects occur, such as color balance is deteriorated and contrast deterioration occurs.
Furthermore, in the captured image using a solid-state image-capturing device, there is a problem in that color hue rotation such that colors are represented by a color differing from the original one in a high-luminance portion when overexposure occurs. For example, when the amount of signal electric charge corresponding to the amount of incident red light among the incident light exceeds the saturation electric charge of a photodiode, colors reproduced from each of the amount of signal electric charge of green and blue and the amount of signal electric charge (amount of saturation electric charge) of red, which do not reach the amount of saturation electric charge, become colors such that the degree of redness is insufficient. In particular, since the human eye is sensitive to flesh color, yellowing of flesh color becomes easily recognizable when flesh color is image-captured in overexposure.
As a method for avoiding this color hue rotation, for example, a method is considered in which levels of the color-difference signals are suppressed in a high-luminance portion where the image color rotation begins to occur. This is a technology that is commonly called “achromatization”, and a process for erasing colors in a step-like manner when the luminance signal level reaches a threshold value or higher and causing the colors to be blown out to white is performed. However, since colors are lost, a problem occurs in that the image appears to have blown out to white.
As another method of avoiding color hue rotation, for example, clipping is performed so that a color-difference signal does not rotate to a signal exceeding the color region after an RGB signal is color-space-converted into a luminance signal and color-difference signals (Y/Cr/Cb) or further into sRGB (standard RGB). This method is well-worn means called a “3D look-up table process”, and the circuit size becomes considerably large.
The color hue rotation avoidance methods of the related art have the following problems that blown out to white occurs due to insufficient gradation in a high-luminance portion and color rotation as a result of an RGB signal clip occurring due to γ correction before the RGB signal is color-space-converted into a luminance signal and color-difference signals (Y/Cr/Cb) cannot be suppressed.
The problem of color hue rotation occurs because the amount of saturation electric charge of pixels is insufficient and the dynamic range is insufficient. The resolving power of digital cameras is nearing that of silver-halide photographs due to a marked increase in the number of pixels in digital cameras, but images obtained using digital cameras are inferior to silver-halide photographs in terms of color reproducibility.
Most digital cameras are provided with a function of selecting a sensitivity condition corresponding to the ISO sensitivity in silver-halide cameras. That is, a sensitivity improvement technology of gain increase is used to capture an image of a low-illumination subject for which a sufficient sensitivity cannot still be obtained by exposure correction. Gain increase enables sensitivity to be increased by amplifying an output signal of a CCD or the like and by relatively increasing the luminance range, making it possible to capture an image of a dark place and a low-illumination subject. ISO 100 is set as a standard sensitivity, and ISO 200, ISO 400, and so on are provided as high sensitivity modes.
A signal processing range is often not appropriately set with respect to sensor output for each setting of the ISO sensitivity of a digital camera, and the output signal range of a fixed image-capturing device, which can be used originally, is not effectively utilized, which is problematical. In the following, this point will be considered.
When the upper limit of a used signal amplitude at ISO 100, that is, at the time of normal image capturing in which a standard gain is used, is denoted as s and the gain at this time is denoted as a, the upper limit of the used signal amplitude at ISO 200, that is, at the time of gain increase, is s/2, and the gain is 2a. In the gain-increased high sensitivity mode, since the amount of exposure is set to be lower than standard exposure, the margin until the amount of saturation electric charge is reached becomes large. Therefore, by expanding the image-capturing range on the high-luminance side of a subject whose image can be captured, signal components of the subject on the high-luminance side, which cannot be obtained in the usual image-capturing mode, will fit into the output signal of that CCD or the like with certainty.
The current situation is that there is no margin from the average amount of exposure of a subject to the saturation level of a CCD. Therefore, in general, the saturation level of a CCD is set as the standard gain, that is, the upper limit of used signal amplitude at ISO 100. For example, if AD conversion is performed at 10 bits on CCD sensor output of the amount of saturated signal (maximum output) 500 mV, at ISO 100, all 500 mV is used as 1023 gradations (refer to FIG. 17). In the case of JPEG (Joint Picture Experts Group), an RGB image signal of 1023 gradations is reassigned to 255 gradations by γ correction. On the other hand, at ISO 200, sensor output 250 mV, which is half of 500 mV, is set as 1023 gradations (refer to FIG. 18), and a signal is clipped at 1023 gradations at the time of gain amplifier output. Therefore, it follows that only half of the CCD maximum output, that is, the amount of saturation electric charge, is used.
However, since the region from 250 mV up to 500 mV is effective as CCD sensor output, it should be utilized as an image-capturing range on the high-luminance side. However, when gradation conversion is performed even in the high sensitivity mode using a γ correction circuit in the same manner as in the standard mode, the image-capturing range of the subject on the high-luminance side, for which the gradation can be converted, by using the image-capturing range limit (in the example shown in FIG. 17, 1023 gradations) in the standard mode as a reference, is limited, and the expansion effect of the image-capturing range on the high-luminance side, which is expanded in the high sensitivity mode, cannot be reflected in the image.
In contrast, there has been proposed an image-capturing apparatus that performs gain increase and changing of gradation conversion characteristics in combination, that is, performs conversion of gradation conversion characteristics such that the effective maximum input value of gradation conversion characteristics is set to be high at the time of a high gain, thereby reflecting the expansion effect of the image-capturing range of an image-capturing device on the high-luminance side, which occurs due to gain increase, in the image, and can thus expand the reproduction region of the subject on the high-luminance side (refer to, for example, Patent Document 3).
According to the image-capturing apparatus, at ISO 200, all 500 mV is used as 2047 gradations, and an RGB image signal of 2048 bits can be reassigned to 255 gradations in accordance with “γ characteristics at the time of a wide D range” by γ correction. The “γ characteristics at wide D range time” are set in such a manner that they are the same characteristics as the standard γ characteristics in the region on the low-luminance side up to 75% of the amount of saturation electric charge and the degree of compression of gradation is high in the region on the high-luminance side higher than 75% of the amount of saturation electric charge.
However, in the method according to Patent Document 3 or the like, it is undeniable that reproducibility in the high-luminance region in which the degree of compression of gradation is increased is sacrificed. If, in particular, gradation conversion is performed on an RGB signal so as to fit into the specified gradation by γ correction before the RGB signal is color-space-converted into a luminance signal and color-difference signals (Y/Cr/Cb), it is not possible to sufficiently suppress color rotation and color overexposure as a result of the occurrence of an RGB signal clip. In Patent Document 3, a method of improving the dynamic range has been disclosed, but no mention has been made in the problem of color hue rotation in the vicinity of a saturation level of color-difference signals.    [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2003-179809    [Patent Document 2] Japanese Unexamined Patent Application Publication No. 2004-23605    [Patent Document 3] Japanese Unexamined Patent Application Publication No. 2002-33956 (FIG. 2, paragraph 0036)